en 

o 

(N 

u 

Oh' 
< 
00 



U 

d 



> 

(N 

o 



X 



The correlation structure of local cortical networks intrinsically results from 

recurrent dynamics 



Moritz Helias^, Tom TetzlaflF^, Markus Diesmann 



1,2 



^Inst. of Neuroscience and Medicine (INM-6) and Inst, for Advanced Simulation 
(IAS-6), Jiilich Research Centre and JARA, Jiilich, Germany 

^Medical Faculty, RWTH Aachen University, Aachen, Germany 



Correspondence to: Dr. Moritz Helias 

tel: +49-2461-61-9467 
m.helias@fz-juelich.de 



Abstract 

Correlated neuronal activity is a natural consequence of network connectivity and shared inputs to pairs 
of neurons, but the task-dependent modulation of correlations in relation to behavior also hints at a 
functional role. Correlations influence the gain of postsynaptic neurons, the amount of information 
encoded in the population activity and decoded by readout neurons and synaptic plasticity. Further, 
it affects the power and spatial reach of extracellular signals like the local-field potential. A theory of 
correlated neuronal activity accounting for recurrent connectivity as well as fluctuating external sources 
is currently lacking. In particular, it is unclear how the recently found mechanism of active decorrelation 
by negative feedback on the population level affects the network response to externally applied correlated 
stimuli. Here, we present such an extension of the theory of correlations in stochastic binary networks. 
We show that (1) for homogeneous external input, the structure of correlations is mainly determined 
by the local recurrent connectivity, (2) homogeneous external inputs provide an additive, unspecific 
contribution to the correlations, (3) inhibitory feedback effectively decorrelates neuronal activity, even 
if neurons receive identical external inputs, and (4) identical synaptic input statistics to excitatory and 
to inhibitory cells increases intrinsically generated fluctuations and pairwise correlations. We further 
demonstrate how the accuracy of mean-field predictions can be improved by self-consistently including 
correlations. As a byproduct, we show that the cancellation of correlations between the summed inputs 
to pairs of neurons does not originate from the fast tracking of external input, but from the suppression 
of fluctuations on the population level by the local network. The suppression of fluctuations on the 
population level is a necessary constraint, but not sufficient to determine the structure of correlations. 
Therefore, the structure of correlations does not follow from the fast tracking of external inputs. 

Author summary 

The co-occurrence of action potentials of pairs of neurons within short time intervals is known since 
long. Such synchronous events can appear time-locked to the behavior of an animal and also theoretical 
considerations argue for a functional role of synchrony. Early theoretical work tried to explain correlated 
activity by neurons transmitting common fluctuations due to shared inputs. This, however, overestimates 
correlations. Recently the recurrent connectivity of cortical networks was shown responsible for the 
observed low baseline correlations. Two different explanations were given: One argues that excitatory 
and inhibitory population activities closely follow the external inputs to the network, so that their effects 
on a pair of cells mutually cancel. Another explanation relies on negative recurrent feedback to suppress 
fluctuations in the population activity, equivalent to small correlations. In a biological neuronal network 
one expects both, external inputs and recurrence, to affect correlated activity. The present work extends 
the theoretical framework of correlations to include both contributions and explains their qualitative 
differences. Moreover the study shows that the arguments of fast tracking and recurrent feedback are not 
equivalent, only the latter correctly predicts the cell-type specific correlations. 

1 Introduction 

The spatio-temporal structure and magnitude of correlations in cortical neural activity have since long 
been subject of research for a variety of reasons: The experimentally observed task-dependent modulation 
of correlations points at a potential functional role. In the motor cortex of behaving monkeys, for example, 
synchronous action potentials appear at behaviorally relevant time points [1]. The degree of synchrony 
is modulated by task performance, and the precise timing of synchronous events follows a change of the 
behavioral paradigm after a phase of re-learning. In primary visual cortex, saccades (eye movements) 
are followed by brief periods of synchronized neural firing [2, 3]. Further, correlations and fluctuations 
depend on the attentive state of the animal [4] , with higher correlations and slow fluctuations observed 
during quiet wakefulness, and faster, uncorrelated fluctuations in the active state [5]. It is still unclear 
whether the observed modulation of correlations is in fact employed by the brain, or whether it is merely 
an epiphenomenon. Theoretical studies have suggested a number of interpretations and mechanisms of 



how correlated firing could be exploited: Correlations in afferent spike-train ensembles may provide a 
gating mechanism by modulating the gain of postsynaptic cells (for a review, see [6]). Synchrony in 
afferent spikes (or, more generally, synchrony in spike arrival) can enhance the reliability of postsynaptic 
responses and, hence, may serve as a mechanism for a reliable activation and propagation of precise 
spatio-temporal spike patterns [7, 8, 9, 10]. Further, it has been argued that synchronous firing could be 
employed to combine elementary representations into larger percepts [11, 12, 7, 13, 14]. While correlated 
firing may constitute the substrate for some en- and decoding schemes, it can be highly disadvantageous 
for others: The number of response patterns which can be triggered by a given afferent spike-train 
ensemble becomes maximal if these spike trains are uncorrelated [15]. In addition, correlations in the 
ensemble impair the ability of readout neurons to decode information reliably in the presence of noise 
(see e.g. [16, 15, 17]). Recent studies have indeed shown that biological neural networks implement 
a number of mechanisms which can efficiently decorrelate neural activity, such as the nonlinearity of 
spike generation [18], synaptic variability and failure [19, 20], short-term synaptic depression [20], and 
the recurrent network dynamics [21, 22, 17]. To study the significance of experimentally observed task- 
dependent correlations, it is essential to provide adequate null hypotheses: Which level and structure 
of correlations is to be expected in the absence of any task-related stimulus or behavior? Even in the 
simplest network models without time varying input, correlations in the neural activity emerge as a 
consequence of shared input [23, 24, 25] and recurrent connectivity [22, 26, 17, 27, 28]. Irrespective 
of the functional aspect, the spatio-temporal structure and magnitude of correlations between spike 
trains or membrane potentials carry valuable information about the properties of the underlying network 
generating these signals [24, 26, 29, 27, 28] and could therefore help constraining models of cortical 
networks. Further, the quantification of spike-train correlations is a prerequisite to understand how 
correlation sensitive synaptic plasticity rules, such as spike-timing dependent plasticity [30], interact with 
the recurrent network dynamics [31]. Finally, knowledge of the expected level of correlations between 
synaptic inputs is crucial for the correct interpretation of extracellular signals like the local-field potential 
(LFP) [32]. 

Previous theoretical studies on correlations in local cortical networks provide analytical expressions 
for the magnitude [25, 22, 17] and the temporal shape [33, 34, 27, 28] of average pairwise correlations, 
capture the influence of the connectivity on correlations [35, 36, 26, 29, 27, 37], and connect oscillatory 
network states emerging from delayed negative feedback [38] to the shape of correlation functions [28] . We 
have in particular shown recently that negative feedback loops, abundant in cortical networks, constitute 
an efficient decorrelation mechanism and therefore allow neurons to fire nearly independently despite 
substantial shared presynaptic input [17] (see also [35, 22, 39]). We further pointed out that in networks 
of excitatory (E) and inhibitory (I) neurons, the correlations between neurons of different cell type 
(EE, EI, II) differ in both magnitude and temporal shape, even if excitatory and inhibitory neurons have 
identical properties and input statistics [17, 28]. It remains unclear, however, how this cell-type specificity 
of correlations is affected by the connectivity of the network. 

The majority of previous theoretical studies on cortical circuits is restricted to local networks driven 
by external sources representing thalamo-cortical or cortico-cortical inputs (e.g. [40, 41, 42]). Most of 
these studies emphasize the role of the local network connectivity (e.g. [43]). Despite the fact that inputs 
from remote (external) areas constitute a substantial fraction of all excitatory inputs (about 50% [7], see 
also [44, 45]), their spatio-temporal structure is often abstracted by assuming that neurons in the local 
network are independently driven by external sources. A priori, this assumption can hardly be justified: 
Neurons belonging to the local cortical network receive, at least to some extent, inputs from identical or 
overlapping remote areas, for example due to patchy (clustered) horizontal connectivity [46, 47]. Hence, 
shared-input correlations are likely to play a role not only for local but also for external inputs. Coherent 
activation of neurons in remote presynaptic areas constitutes another source of correlated external input, 
in particular for sensory areas [48, 5, 49, 4]. So far, it is largely unknown how correlated external input 
affects the dynamics of local cortical networks and alters correlations in their neural activity. 

In this article, we investigate how the magnitude and the cell-type specificity of correlations depend 
on i) the connectivity in local cortical networks of finite size and ii) the level of correlations in external 
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Figure 1: State transitions of a binary neuron. Each neuron is updated at random time points, intervals 
are i.i.d. exponential with mean duration r, so the rate of updates per neuron i is r^^. The probability 
of neuron i to end in the up-state (1) is determined by the gain function Fi{n) which potentially depends 
on the states n of all neurons in the network. The up-transitions are indicated by black arrows. The 
probability for the down state (0) is given by the complementary probability 1 — Fi{n), indicated by gray 
arrows. 

inputs. Existing theories of correlations in cortical networks are not sufficient to address these questions 
as they either do not incorporate correlated external input [33, 17, 27, 26, 29] or assume infinitely large 
networks [22]. Lindner et al. [35] studied the responses of finite populations of spiking neurons receiving 
correlated external input, but described inhibitory feedback by a global compound process. 

Our work builds on the existing theory of correlations in stochastic binary networks [33], a well- 
established model in the neuroscientific community [50, 22]. This model has the advantage of requiring for 
its analytical treatment elementary mathematical methods only. We employ the same network structure 
used in the work by Renart et al. [22] which relates the mechanism of recurrent decorrelation to the 
fast tracking of external signals (see [51] for a recent review). This choice enables us to reconsider the 
explanation of decorrelation by negative feedback [17], originally shown for networks of leaky integrate- 
and-fire neurons, and to compare it to the findings of Renart et al. In fact, the motivation for the choice 
of the model arose from the review process of [17], during which both the reviewers and the editors 
encouraged us to elucidate the relation of our work to the one of Renart et al. in a separate subsequent 
manuscript. The present work delivers this comparison. 

The remainder of the manuscript is organized as follows: In "Methods" , we develop the theory of cor- 
relations in recurrent random networks of excitatory and inhibitory cells driven by fluctuating input from 
an external population of finite size. We present a method accounting for the fiuctuations in the synaptic 
input to each cell, which effectively linearize the hard threshold of the neurons. We further include the 
resulting finite-size correlations into the established mean-field description [50, 52] to increase the accuracy 
of the theory. In "Results", we first show in "Correlations are driven by intrinsic and external fluctuations" 
that correlations in recurrent networks are not only caused by the externally imposed correlated input, 
but also by intrinsically generated fiuctuations of the local populations. We demonstrate that the external 
drive causes an overall shift of the correlations, but that their relative magnitude is mainly determined 
by the intrinsically generated fluctuations. In "Cancellation of input correlations" , we revisit the 
earlier reported phenomenon of the suppression of correlations between input currents to pairs of cells [22] 
and show that it is a direct consequence of the suppression of fluctuations on the population level [17]. 
Subsequently, in "Influence of connectivity on the correlation structure" , we investigate in how 
far the reported structure of correlations is a generic feature of balanced networks and isolate parameters 
of the connectivity determining this structure. Finally, in "Discussion", we summarize our results and 
their implications for the interpretation of experimental data, discuss the limitations of the theory, and 
provide an outlook of how the improved theory may serve as a further building block to understand 
processing of correlated activity. 

2 Methods 



2.1 Networks of binary neurons. The state of a binary neuron is either or 1, where 1 indi- 
cates activity, inactivity [33, 53, 22]. The model shows stochastic transitions (at random points in 
time) between these two states controUed by transition probabihties, as iUustrated in Figure 1. Using 
asynchronous update [54], in each infinitesimal interval [t,t + St) each neuron in the network has the 
probability -St to be chosen for update [55], where t is the time constant of the neuronal dynamics. An 
equivalent implementation draws the time points of update independently for all neurons. For a partic- 
ular neuron, the sequence of update points has exponentially distributed intervals with mean duration 
r, i.e. update times form a Poisson process with rate t~^. We employ the latter implementation in the 
globally time-driven [56] spiking simulator NEST [57], and use a discrete time resolution h = 0.1ms for 
the intervals. The stochastic update constitutes a source of noise in the system. Given the i-th neuron 
is selected for update, the probability to end in the up-state {rii — 1) is determined by the gain function 
Fi{n) which possibly depends on the activity n of all other neurons. The probability to end in the down 
state {ui = 0) is 1 — Fi{n). This model has been considered earlier [58, 33, 53], and here we follow the 
notation introduced in the latter work. 

The state of the network of A^ such neurons is described by a binary vector n — (rii, . . . , un) G {0, 1}^. 
The stochastic system is completely characterized by the joint probability distribution p{n) in all N 
binary variables n. Knowing the joint probability distribution, arbitrary moments can be calculated, 
among them pairwise correlations. Here we are only concerned with the stationary state of the network. 
A stationary solution of p{n) implies that for each state a balance condition holds, so that the incoming 
and outgoing probability fluxes sum up to zero. The occupation probability of the state is then constant. 
We denote as n^^ = (ni, . . . ,ni_i, l,rii+i, . . . ,71^) the state, where the i-th neuron is active (n^ = 1), 
and ni_ where neuron i is inactive {rii ~ 0). Since in each infinitesimal time interval at most one neuron 
can change state, for each given state n there are A^ possible transitions (each corresponding to one of 
the A^ neurons changing state). The sum of the probability fluxes into the state and out of the state must 
compensate to zero [59], so 



= r^ = E 2^^^ I Mn.-)^.(n._) -p(n,+ )(l - ^,(n,+ )) | V n e {0, 1}^. (1) 

direction of flux ^neuron i transition up neuron i transition downy 

From this equation we derive expressions for the first (rife) and second moments {ukni) by multiplying 
with Ukfii and summing over all possible states n e {0, 1}^, which leads to 

N 

= Y. ^"fc»i(2»» - l) fe(n,_)f,(n,,) -p(n,+ )(l - f»(n,+ )))^ . 

Note that the term denoted Gi{n\ni) does not depend on the state of neuron i. We use the notation 
n\ni for the state of the network excluding neuron i, i.e. n\ni — (ni, . . . , n^-i, rii^i, . . . , hn). Separating 
the terms in the sum over i into those with i ^ k,l and the two terms with i ~ k and i — I, we obtain 



N 

= ^ ^ nkni{2ni-l)Gi{n\n.,)+nkni{2nk-l)Gk{n\nk) + nkni{2ni-l)Gi{n\ni) 

n i=l.i^k,l 

N 

= ^ ^ nfenz(G.,(n\ni) -Gi(n\n.,)) +^nfcn;Gfe(n\nfe) -f ^nfen/G'/(n\n;) 

where we obtained the first term by explicitly summing over state ni e {0, 1} (i.e. using X^nslo i>™ ~ 
SnXniSlo i}«-i X]ni=o ^^^ evaluating the sum X]ni=o)- This first sum obviously vanishes. The remaining 
terms are of identical form with the roles of k and I interchanged. We hence only consider the first of 



them and obtain the other by symmetry. The first term simphfies to 
^TifeTi; Gk{n\nk) ''= ^ ni Gk{n\nk) 

n n\rifc 

dcf. Gfc \T,n\nkPi^k-)Fk{nk-)+p{nk+)Fk{nk+)~p{nk+) ioi k ^ I 

J2n\nk Pi^k-) ni Fk{nk-) + p{nk+) ni Fk{nk+) -nip{nk+) for k ^ I 

(Ffc(n)> - (uk) iork^l 

(Ffc(n) ni) - {nkfii) ior k ^ I 

where we denote as (/(n)) = X^nelo i>"P('^)/('^) ^^e average of a functfon /(n) with respect to the 
distribution p{n). Taken together with the mirror term k ■<^ I, we arrive at two conditions, one for the 
first {k = I, (ni) = (uk)) and one for the second (fc 7^ I) moment 

, 2(Ffc(n)) ioik^l 

2{nkni) ^ <^ ^ ^^ ^^ . (2) 

{Fk{n)ni) + {Fi{n)nk) ioi k ^ I 

Considering the covariance Cki — (dnkSni) with centrahzed variables Suk — rik — (rik), for k 7^ / one 
arrives at 

2cki = {Fkin)Sni) + {Fi{n)Snk). (3) 

This equation is identical to eq. 3.9 in [33], to eqs. 3.12 and 3.13 in [53], and to eqs. (19)-(22) in [22, 
Supplementary material] . 

2.2 Mean-field solution. Starting from (1) for the general case ^l"' ' 7^ 0, a similar calculation as 
the one resulting in (2) for k — I leads to 

T^K) - {Fk{n))~{nk}, 

where we used (n|) — (rik), valid for binary variables. As in [22] we now assume a particular form for 
the gain function and for the coupling between neurons by specifying 

Ffc(n) - H{hk-e) 

N 

hk = }^Jkini 

\l if .T > 

H{x) = I 

[0 ifx<0 

where Jki is the incoming synaptic weight from neuron I to neuron fc, H is the Heaviside function, 
and 6 is the threshold of the activation function. We denote by hk the summed synaptic input to the 
neuron, sometimes also called the "field". Because n^. = n^, the variance ak of a binary variable is 
afc = {n\) — (uk)^ = (1 — {nk)){nk). We now aim to solve (2) for the case k = I, i.e. the equation 
{nk) — {Fk}- In general, the right hand side depends on the fluctuations of all neurons projecting to 
neuron k. An exact solution is therefore complicated. However, for sufficiently irregular activity in the 
network we assume the neurons to be approximately independent. Further assume that in a network of 
homogeneous populations a (same parameters r, 9 and same statistics of the incoming connections for 
all neurons, i.e. same number Kai3 and strength Ja/3 of incoming connections from neurons in a given 
population (3) the mean activity of an individual neuron can be represented by the population mean 



m-a — (t^ Sigct ""«)• "^^^ mean input to a neuron in population a then is 

(/la) = '^Kal3Jal3mp = ^ia- (4) 

We assumed in the last step identical synaptic amplitudes Jap for a synapse from a neuron in population 
/3 to a neuron in population a. So the input to each neuron has the same mean {ha). As a first 
approximation, if the mean activity in the network is not saturated, i.e. neither nor 1, mapping this 
activity back by the inverse gain function to the input, ha must be close to the threshold value, so 

{ha) - 0. (5) 

This relation may be solved for m^ and mj to obtain a coarse estimate of the activity in the network 
[50, 52]. In mean- field approximation we assume that the fluctuations of the fields of individual neurons 
ha around their mean are mutually independent, so that the fluctuations 6ha — ha — (ha) of ha are, in 
turn, caused by a sum of independent random variables and hence the variances add up to the variance 
ffa of the field 

(Shi) = ^ iS:a/3Ja;3m^(l- 771^) = Cr^. (6) 

As ha is a sum of typically thousands of synaptic inputs, it approaches a Gaussian distribution ha ^ 
Af{fJ,a, cTa) with mean Ha and variance a a- In this approximation the mean activity in the network is the 
solution of 

d 

T-^ma+ma == {Fa{mE,mi,mj;)) Va €{£',/} 



dt 



H{x — 6) Af{na,<^aj ^) dx 



7V(^a,cr„,x)dx 

= Ifcf^V (7) 

This equation needs to be self-consistently solved with ^g|^ = by numerical or graphical methods in 
order to obtain the stationary activity, because ^a{mE, mi, rrix) and aairriE, mi, fUx) depend on vricSa € 
{E,I,X} themselves. We here employ the algorithm hybrd and hybrj from the MINPACK package, 
implemented in scipy (version 0.9.0) [60] as the function scipy. optimize. fsolve. 

2.3 Linearized equation for correlations and susceptibility. In general, the term {Fk{n) 6ni) 
in (3) couples moments of arbitrary order, resulting in a moment hierarchy [53]. Here we only determine 
an approximate solution. Since the single synaptic amplitudes Jki are small, we linearize the effect of a 
single synaptic input. We apply the linearization to the two terms of the form (^^(n) Sni) on the right 
hand side of (3). In the recurrent network, the activity of each neuron in the vector n may be correlated 
to the activity of any other neuron n^. Therefore, the input hk sensed by neuron k not only depends 
on ni directly, but also indirectly through the correlations of ni with any of the other neurons rii that 
project to neuron k. We need to take this dependence into account in the linearization. Considering the 
effect of one particular input rii explicitly one gets 



{Fkin)6ni) - {H{hk~9)6ni) 

= (i7(/ife\„^ + Jki - 0) UiSni + H{hk\n, - 6*) (1 - Ui) Sni) 

= ((iJ(/ifc\„, + Jk^ -0)- H{hk\„^ - e)) nMi) + {H{hk\n^ - e) 6ni). 



The first term ((-ff(ft-fe\„i + Jki ^ 0) ^ H{hi^\„. — 9)) riiSni) already contains two factors n^ and Sni, so it 
takes into account second order moments. Performing the expansion for the next input would yield terms 
corresponding to correlations of higher order, which are neglected here. This amounts to the assumption 
that the remaining fluctuations in /ife\„; are independent of Ui and ni , and we again approximate them 
by a Gaussian random variable x ~ M{^k,<^k) with mean /ife — (hk) and variance a^ — {Sh'j,), so 
{{H{x + Jki -9) - H{x - 9))).j;{niSni)n '^^ S{^ik,<Jk) Jki {niSni)n + 0{Jl^). Here we used the smallness of 

the synaptic weight Jki and replaced the difference by the derivative Si^k, crk) — j'-^d^k-'k) ^ 

which has the form of a susceptibility. Using the explicit expression for the Gaussian integral (7), the 
susceptibility is exactly 



S{^ik.cJk) = -=^e -^ . (8) 

{fli for i — I 
, the equation 
Cii ioi i y^l 

for the pairwise correlations (3) in linear approximation takes the form 

2cki = S{iik,(Tk) l^^JkjCji^ Jkiai\ + S{fjii,ai) \^^JijCjk + Jikak\, (9) 

corresponding to eq. (6.8) in [33] and eqs. (31)-(33) in [22, Supplementary material]. Note, however, 
that the linearization used in [33] relies on the smoothness of the gain function due to additional local 
noise, whereas here and in [22, Supplementary material] a Heaviside gain function is used and only the 
existence of noise generated by the network itself justifies the linearization. If the input to each neuron is 
homogeneous, i.e. ^.k = fJ-a and ak = era for all neurons k in population a, a structurally similar equation 
connects the correlations c^^ = jj-jg- J2kea lep kM '^ki averaged over disjoint pairs of neurons belonging 
to two (possibly identical) populations a, (3 with the population averaged variances a^ = -^ X^fcea ^^ 

7G{B,/,X} ^ " 

with Wa/3 = S{^la,a-a)JapKai3- 

In deriving the last expression, we replaced variances of individual neurons and correlations between 
individual pairs by their respective population averages and counted the number of connections. This 
equation corresponds to eqs. (9.14)-(9.16) in [33] (which lack, however, the external population X, and 
note the typo in the first term in line 2 of 9.16, which should read — ^Jb/C//(0)) and eqs. (36) in [22, 
Supplementary material]. Written in matrix form (10) reads 



(11) 





Nx 



resulting in (19) of the main text, where we defined 



2 — 2wee -'^wei 

A = I -wiE 2 - {wee + wii) -Wei 
-2wiE 2 - 2w// 

2wEE \ / 2w£x 

B = I w/B wb/ I C = 

2w// 



Wix WEX 

V 2w/x 



D = 2--^^ --i./ ] £;^r"^^ ). (12) 

V -W/B 2-wii \ Wix / 




^- (13) 



The explicit solution of the latter system of (11) is 

_ 1 I (^ ^ Wjj)wEX + WElWix 

(2 - Wee){2 - Wii) - WElWiE \ (2 - wee)wix + wiewex 

2.4 Mean-field theory including finite-size correlations. The mean-field solution presented in 
"Mean-field solution" assumes that correlations among the neurons in the network are negligible. This 
assumption enters the expression (6) for the variance of the input to a neuron. Having determined the 
actual magnitude of the correlations in (11), we are now able to state a more accurate approximation in 
which we take these correlations into account, modifying the expression for the variance of the field ha 

al = ^ KaisJl/^rn/^il-mp) + ^ {KJ)ap[KJ)a-iCp^ (14) 

0£{EJ,X} p,-i£{E,I,X} 

with {KJ)afi = KapJaP- 

This correction suggests an iterative scheme: Initially we solve the mean- field equation (7) assuming Cap = 
(hence a a given by (6)). In each step of the iteration we then calculate the correlations by (11), compute 
the mean- field solution of (7) and the susceptibility S{^a,<^a) (8), taking into account the correlations (14) 
determined in the previous step. These steps are iterated until the solution {ma,Cai3 ^ol^P) converges. 
We use this approach to determine the correlation structure in Figure 3, where we iterated until the 
solution became invariant up to a residual absolute difference of 10~^^. A comparison of the distribution 
of the total synaptic input He at the end of the iteration with a Gaussian distribution with parameters 
^E and ge is shown in Figure 3F. 

3 Results 

Our aim is to investigate the role of recurrence and external input on the magnitude and structure of 
cross correlations between the activities in a recurrent random network. Denoting the activity of neuron 
i as ni[t) and its mean activity as mi — {ni{t))t, the (zero time lag) covariance of the activities of a pair 
{i,j) of neurons is defined as Cy = {5ni{t)5nj{t)) t, where 5ni{t) — ni{t) — mi is the deviation of neuron 
i's activity from expectation and the average ()f is over time and realizations of the stochastic activity. 
We employ the established recurrent neuronal network model of binary neurons in the balanced regime 
[50]. The binary dynamics has the advantage to be more easily amendable to analytical treatment than 
spiking dynamics. A method to calculate the pairwise correlations exists since long [33]. The choice of 
binary dynamics moreover renders our results directly comparable to the recent findings on decorrelation 
in such networks [22]. Our model consists of three populations of neurons, one excitatory and one 
inhibitory population which together represent the local network, and an external population providing 
additional excitatory drive to the local network, as illustrated in Figure 2. The external population 
may either be conceived as representing input into the local circuit from remote areas or as representing 
sensory input. The external population contains Nx neurons, which are pairwise uncorrelated and have a 
stochastic activity with mean mx- Each neuron in population a e {-E, /} within the local network draws 
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Figure 2: Recurrent local network of two populations of excitatory [E) and inhibitory (/) neurons 
driven by a common external population {X). The external population X delivers stochastic activity 
to the local network. The local network is a recurrent Erdos-Renyi random network with homogeneous 
synaptic amplitudes of weight J^p coupling a neuron in population /3 to a neuron in population a, for 
a, /3 e {E, /} and same parameters for all neurons. There are N = 8192 neurons in both the excitatory 
and the inhibitory population. The connection probability is p — 0.2, and each neuron in population 
a receives the same number K — pN of excitatory and inhibitory incoming synapses. The size Nx of 
the external population determines the amount of shared input received by each pair of cells in the local 
network. The neurons are modeled as binary units with a hard threshold 6. 

K = pN connections randomly from the finite pool of Nx external neurons. Nx therefore determines 
the number of shared afferents received by each pair of cells from the external population with on average 
K"^ /Nx common synapses. In the extreme cases Nx = K all neurons receive exactly the same input, 
for large Nx -^ oo the fraction of shared external input approaches 0. The common fluctuating input 
received from the finite-sized external population hence provides a signal imposing pairwise correlations, 
the amount of which is controlled by the parameter Nx ■ 

3.1 Correlations are driven by intrinsic and external fluctuations. To explain the correlation 
structure observed in a network with external inputs (Figure 2), we here extend the existing theory of 
pairwise correlations [33] to include the effect of externally imposed correlations. The global behavior of 
the network can be studied with help of the mean-field equation (7) for the population-averaged mean 
activity m^ = N~'^ J^ziK) 



= 2°^* ' ^ ' ^ $(Ma,Cr„), 






(15) 



where the fluctuations of the input ha to a neuron in population a are to good approximation Gaussian 
with the moments 



ia — {ha) — } ^KapJg 



pmp 



(16) 



c^a = (Shi) = Y^ KafsJlpmpil - mp). 



In order to determine the average activities in the network, the mean-field equation (15) needs to be 
solved self-consistently, as the right hand side depends on the mean activities TOq, through (16), as 
explained in "Mean-field theory including finite-size correlations". HeieKa^i denotes the number 
of connections from population /? to a, and Ja^ their average synaptic amplitude. Once the mean activity 
in the network has been determined, we can ask for the structure of correlations. For simplicity we focus 
on the zero time lag correlation, Cy = {6ni(t)6nj{t))t, where Sni{t) = ni(t) — {ni)t is the deflection of 



neuron Vs activity from baseline and a^ = {Snf{t))t 



t(l — {ni)t) is the variance of neuron i's activity. 



Starting from the master equation for the network of binary neurons, in "Methods" for completeness 
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Figure 3: Correlations in a network of three populations as illustrated in Figure 2 in dependence of the 
size Nx of the external population. Each neuron in population a € {E, 1} receives pN randomly drawn 
excitatory inputs with weight JaE = -7^ , pN randomly drawn inhibitory inputs of weight Jai — — 7= 
and pN external inputs of weight Jax ~ -7^ (homogeneous random network with fixed in-degree). A 
Correlations averaged over pairs of neurons within the local network (17). Dots are results of direct 
simulation over T = 30 s averaged over (N/2)'^ pairs of neurons and curves show the analytical result 
(19). The point "DC" shows the correlation structure emerging if the drive from the external population 
is replaced by a constant value with the same mean. B Correlations between neurons within the local 
network and the external population averaged over pairs of neurons (same labeling as in A) . C Correlation 
between the inputs to a pair of cells in the network decomposed into the contributions due to shared 
inputs Cshared (gray, eq. 20) and due to correlations Ccorr (light gray, eq. 21). Dashed curves and 
St. Andrew's Crosses show the contribution due to external inputs, solid curves and dots show the 
contribution from local inputs. The sum of all components is shown by black dots and curve. Curves are 
theoretical results based on (19), (20), and (21), symbols are obtained from simulation. D Probability 
distribution of the fluctuating input He to a single neuron in the excitatory population. Dots show 
the histogram obtained from simulation binned over the interval [min(/i£;),max(/i£;)] with a bin size of 
—2Jai. The gray curve is the prediction of a Gaussian distribution obtained from mean- field theory 
neglecting correlations, with mean and variance given by (4) and (6), respectively. The black curve takes 
into account correlations in the afferent signals and has a variance given by (14). Other parameters: 
simulation resolution At = 0.1ms, synaptic delay d = At, activity measurement in intervals of 1ms. 
Threshold of the neurons 9 = 1, time constant of inter- update-intervals r = 10 ms. The average activity 
in the network is tue — Tnj ~ mx = 0.5. 
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and consistency in notation we re-derive the self-consistent equation that connects the cross covariances 
Cq/3 averaged over pairs of neurons from population a and /3 and the variances aa averaged over neurons 
from population a 



The obtained inhomogeneous system of linear equations (19) reads [33] 



^ E -- (17) 





2cq/3 — -rj-Wajsafj + y. WajC^ p + t'ra,nspose{a -ir^ f3). (18) 

^ -f€{E,I,x} 

Here Wajs measures the effective linearized coupling strength from population /3 to population a that 
depends on the number of connections Kap from population (3 to a, their average synaptic amplitude 
Jai3 and the susceptibility Sa of neurons in population a as Wa/a — S{fj,a,o'a) Kap Jap- The susceptibility 
S{iia^aa) given by (8) quantifies the influence a fluctuation in the input to a neuron in population a 
has on the output. It depends on the working point {p.a,<^a) of the neurons in population a. The 
autocorrelations a^, a/ and ax are the inhomogeneity in the system of equations, so they drive the 
correlations, as pointed out earlier [33]. This is in line with the linear theories [61, 28] for leaky integrate- 
and-fire model neurons, where cross-correlations are proportional to the auto-correlations. The system 
of equations (18) is identical to [33, eqs. (9.14)-(9.16)]. Note that this description holds for finite-sized 
networks. With the symmetry cei = cie, (18) can be written in matrix form as 



(19) 



Nx 

with the explicit forms of the matrices A, . . . ,E given in (12). This system of linear equations can be 
solved by elementary methods. From the structure of the equations it follows, that the correlations 
between the external input and the activity in the network, cex and cjx , are independent of the other 
correlations in the network. They are solely determined by the solution of the system of equations in the 
second line of (19), driven by the fluctuations of the external drive ax/Nx- The correlations among the 
neurons within the network are given by the solution of the first system in (19). They are hence driven 
by two terms, the fluctuations of the neurons within the network proportional to ue/Ne and aj/Nj and 
the correlations between the external population and the neurons in the network, cex and cjx- 

The second line of (19) shows that all correlations depend on the size Nx of the external population. 
Since the number K = pN of randomly drawn afferents per neuron from this population is constant, 
the mean number of shared inputs to a pair of neurons is K'^ /Nx- In the extreme case Nx = K on 
the left of Figure 3 all neurons receive exactly identical input. If the recurrent connectivity would be 
absent, we would hence have perfectly correlated activity within the local network, the covariance between 
two neurons would be equal to their variance a„ — ma{l — ma), in this particular network a^ ~ 0.25. 
Figure 3A shows that the covariance in the recurrent network is much smaller; on the order of 10""*. The 
reason is the recently reported mechanism of decorrelation [22], explained by the negative feedback in 
inhibition-dominated networks [17]. Increasing the size of the external population decreases the amount 
of shared input, as seen in Figure 3C. In the limit where the external drive is replaced by a constant 
value (shown as point "DC"), the external drive does consequently not contribute to correlations in the 
network. Figure 3 A shows that the relative position of the three curves does not change with Nx ■ The 
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overall offset, however, changes. This can be understood by inspecting the analytical result (19). The 
solution of this system of linear equations is a superposition of two contributions. One is due to the 
externally imposed fluctuations, proportional to ax/Nx the other is due to fluctuations generated within 
the local network, proportional to ux/Ne and ai/Nj. Varying the size of the external population only 
changes the external contribution, causing the variation in the offset, while the internal contribution, 
causing the splitting between the three curves, stays constant. In the extreme case (point DC, ax = 0) 
we still observe a similar structure. The slightly larger splitting is due to the reduced variance ct^ in the 
single neuron input, which consequently increases the susceptibility Sa (8). 

Figure 3D shows the probability distribution of the input ha to a neuron in population a = E. The 
histogram is well approximated by a Gaussian. The first two moments of this Gaussian are /Xq, and tr^ 
given by (16), if correlations among the afferents are neglected. This approximation is shown to deviate 
from the result of direct simulation. Taking into account the correlations among the afferents affects 
the variance in the input according to (14). The latter approximation is a better estimate of the input 
statistics, as shown in Figure 3D. This improved estimate can be accounted for in the solution of the mean- 
field equation (15), which in turn via the susceptibility Sa affects the correlations. Iterating this procedure 
until convergence, as explained in "Mean-field theory including finite-size correlations", yields 
the analytical results presented in Figure 3. 

3.2 Cancellation of input correlations. We would like to understand how the structure of cor- 
relations relates to the earlier report of fast tracking [50, 52]. Small correlations observed in balanced 
recurrent networks were explained by the property of recurrent networks to track their input on a fast 
time-scale [22, their eq. (2)]. Figure 4A shows the population activities in a network of three populations 
for fixed numbers of neurons N^ — Ne — Nj — N and symmetric connectivity as in Figure 3. The 
deflections of the excitatory and the inhibitory population partly resemble those of the external drive to 
the network, but partly the fluctuations seem to be independent. Our theoretical result for the correla- 
tion structure explains this result (19): the fluctuations in the network are not only driven by external 
input (proportional to ax), but are also driven by the fluctuations generated within the local populations 
(proportional to ue and a/). The idea of fast tracking of the external signal was derived from the obser- 
vation that the fluctuations in the population-averaged input ha ~ -j^ X^jPa ^* ^'"^ suppressed [22]. This 
suppression can be observed by decomposing the input ha to the population a into contributions from 
excitatory (including external neurons) and from inhibitory cells, haS — {KJ)aEnE + {KJ)aX'nx and 
hai = {KJ)aini, respectively, where we used the short hand (KJ)ap — KapJajS- As shown in Figure 4E, 
the contributions of excitation and inhibition cancel each other so that the total input fluctuates close 
to the threshold (here 6* = 1) of the neurons: the network is in the balanced state [50]. Moreover, this 
cancellation not only holds for the mean value, but also for fast fluctuations, which are consequently 
reduced in the sum ha compared to the individual components haE and hai (Figure 4E). We will now 
show that this suppression of fluctuations directly implies a relation for the correlation (ShiShj) between 
the inputs to a pair (i, j) of individual neurons. There are two distinct contributions to this correlation 
(ShiShj) — Csharod.Q + Ccorr.Q, ouc duc to common inputs shared by the pair of neurons (both neurons i, j 
assumed to belong to population a) 

Cshared,a = Y. ^^ J)^^^ (20) 

l3e{E.I.X} ^ 

and one due to the correlations between afferents 

Ccorr.Q = 2_/ {KJ)afi{KJ)a^Ci3^. (21) 

Figure 4C shows these two contributions to be of opposite sign but approximately of same magnitude. 
Figure 3C shows a further decomposition of the input correlation into contributions due to the external 
sources and due to connections from within the local network. The sum of all components is much smaller 
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Figure 4: Activity in a network of 3N = 3 x 8192 binary neurons with synaptic amplitudes JaE — JaX — 
5/^/N, Jai — — 10/-\/]V depending exclusively on the type of the sending neuron [E or /). Each neuron 
receives K = pN randomly drawn inputs (fixed in-degree). A Population averaged activity (black E, 
gray /, light gray X). Analytical prediction (5) for the mean activities ttie = mi (dashed horizontal line) 
and numerical solution of mean field equation (7) (solid horizontal line). B Cross covariance between 
excitatory neurons (black), between inhibitory neurons (gray), and between excitatory and inhibitory 
neurons (light gray). Theoretical results (19) shown as dots. St. Andrew's Crosses indicate the theoretical 
prediction of [22, suppl. eqs. 38,39]. C Correlation between the input currents to a pair of excitatory 
neurons. The black curve is the contribution due to pairwise correlations Ccorr, the gray curve is the 
contribution of shared input Cgharcd- The symbols show the theoretical expectation (20) and (21) based 
on [22] (crosses) and based on (19) (dots). D Similar as B, but showing the correlations between external 
neurons and neurons in the excitatory and inhibitory population. Note that both theories yield cex — 
cix, so for each theory ([22] crosses, (19) dots) only the symbol for cex is visible. E Contributions 
hEE (gra-y) due to excitatory synapses and Hei (light gray) due to inhibitory synapses to the input 
He averaged over all excitatory neurons. Duration of simulation T = 100 s, mean activity mx =0.1, 
tue — mj ~ 0.11, other parameters as in Figure 3. 
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than each individual component. This cancellation can be understood from the observation of the small 
fluctuations in the population-averaged input {Sh'^) ~ 0, because 



Oc^{Shl) = {\ Y. iKJ)o.pSnA ) = Y. iKJUp{KJ)o,^{SnpSn^) (22) 

Kl3e{EJ,X} J I l3,ie{E,I,X} 



I3(^{E.,I.X} ^ /3,7G{B,/,X} 



^shared 



Q T '-corr,Q5 



where in the second step we used the general relation between the covariance {SnpSrij) among two 
population averaged signals n^? and n^ and the population-averaged variance a^j and pairwise averaged 
covariances c^^, which reads [17, cf. eq. (1)] 



{SnpSn^) = 




(23) 



Np 



This suppression of fluctuations in the population-averaged input is a consequence of the overall negative 
feedback in these networks: a fluctuation 5ha of the population averaged input ha causes a response 
in network activity which is coupled back with a negative sign, counteracting its own cause and hence 
suppressing the fluctuation 5ha- Expression (22) is an algebraic identity showing that hence also corre- 
lations between the total inputs to a pair of cells must be suppressed. This argument also shows why the 
suppression of input-correlations does not rely on a balance between excitation and inhibition; it can as 
well be observed in purely inhibitory networks [17, cf. text following eq. (21) therein], where the overall 
negative feedback suppresses population fluctuations 5ha in exactly the same manner. 

It is easy to see that the cancellation condition (22) does not uniquely determine the correlations in the 
network. Figure 5 shows the same measures of activity as Figure 4, but for the network connectivity used 
in [22, their Fig. 2]. Except for J// and Jix the parameters are the same as in Figure 4. Moreover, we 
distributed the number of incoming connections K per neuron according to a binomial distribution, as in 
the original publication. The cancellation of fluctuations on the input side is evident from Figure 5C and 
E. Comparing panels B of Figure 4 and Figure 5 we observe that this cancellation, however, does not result 
in the same correlation structure: In Figure 4 the correlations fulfill cee > cei > c//, while in Figure 5 
the relation is cei — cee > cii- This also explains why the attempt to derive the structure of correlations 
from {5h?) ~ fails, as seen in Figure 5B: the predicted correlations [22, Supplementary material, eqs. 
38-39] do not coincide with the results obtained by direct simulation. Note, however, that the wrong 
correlation structure, due to the above discussed relation (22), still fulfills the cancellation condition for 
correlations on the input side, as shown in Figure 5C. The reason for the failure is the assumption that 
fluctuations in the network are exclusively driven by the external input. This assumption [22, eq. (2)] is 
reflected in the expression cei oc ax, which is directly proportional to the external fluctuations. What 
is missing is the contribution due to the intrinsic fluctuations of the local populations {E,I), as they 
appear in (19). This can be shown explicitly by setting ge = and a/ = in (19), resulting in a similar 
prediction for cei, as shown in Figure 5B (plus symbol). The remaining deviation between both theories 
is due to the different susceptibilities S used by the two approaches. Note that the full theory (19) 
predicts the structure of correlations with only minor deviations. In summary, the cancellation condition 
imposes a constraint on the structure of correlations but is not sufficient as a unique determinant. 
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Figure 5: Activity in a network of 3A^ = 3 x 8192 binary neurons as described in [22], with Jee = 5/viV, 
Jei = -10/Vn, Jie = 5/v^, Jii = ~9/VN, Jex = 5/v^, Jix = 4///V. Number K of synaptic 
inputs binomially distributed as iiT ^ B{N,p), with connection probabihty p — 0.2. A Population 
averaged activity (black E, gray /, light gray X). Analytical prediction (5) for the mean activities 
tue = iTT'i (dashed horizontal line) and numerical solution of mean field equation (7) (solid horizontal line). 
B Cross correlation between excitatory neurons (black curve), between inhibitory neurons (gray curve), 
and between excitatory and inhibitory neurons (light gray curve) obtained from simulation. St. Andrew's 
Crosses show the theoretical prediction from [22, suppl. eqs. 38,39] (prediction yields cee = c// = 0, so 
only one cross shown). Dots show the theoretical prediction (19). The plus symbol shows the prediction 
for the correlation cei if the terms proportional to qe and a/ are set to zero. C Correlation between the 
input currents to a pair of excitatory neurons. Contribution due to pairwise correlations Ccorr.B (black 
curve) and due to shared input Csharcd.B (gray curve). Symbols show the theoretical predictions based on 
[22] (crosses) and based on (19) (dots). D Similar as B, but showing the correlations between external 
neurons and neurons in the excitatory and inhibitory population. E Fluctuating input He averaged over 
the excitatory population (black), separated into contributions from excitatory synapses Hee (gray) and 
from inhibitory synapses Hei (light gray). Duration of simulation T ~ 100s, mean activity mx = 0.1, 
niE — "nil — 0.11, other parameters as in Figure 3. 
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Figure 6: Connectivity structure determines correlation structure. In the left column (A,C,E) we vary 
ci = [Jee + Jii)/'2', in the right column (B,D,F) C2 — [Jee ~ Jii)!'^- A,B Mean activity in the network 
as a function of the structural parameters c\ and C2, respectively. C,D Correlations averaged over pairs 
of neurons. Dots obtained from direct simulation, solid curves given by theory (19) E,F Eigenvalues 
(25) of the population-averaged connectivity matrix; solid curves show the real part, dashed curves the 
imaginary part. 
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3.3 Influence of connectivity on the correlation structure. Comparing Figure 4B and Figure 5B, 
tlie structure of correlations is obviously different. In Figure 4B, the structure is cee > cei > c//, whereas 
in Figure 5B the relation is cei — cee > c//. The only difference between these two networks are the 
coupling strengths Ju and Jjx ■ In the following we derive a more complete picture of the determinants 
of the correlation structure. In order to identify the parameters that influence the fluctuations in these 
networks, it is instructive to study the mean-field equation for the population-averaged activities. Lin- 
earizing (15) for small deviations Sua — Ua ~ rria of the population-averaged activity n^ from the fixed 
point rria, for large networks with N > Kap 3> 1 the dominant term is proportional to the change of 
the mean S/ia = '^B{'^-^)a-i3S'>T'Pi because the standard deviation Saa is only proportional to ^J Kap- To 
linear order we hence have a coupled set of two differential equations 




and 5(/iQ,(TQ,) = 



/ WEE WEI 

y wiE wjj 




noise -I- external drive (24) 



The dynamics of this coupled set of linear differential equations is determined by the two eigenvalues of 
the effective connectivity 

Ai,2 = eig{{wap}) 



WEE+Wii fwEE-WjA'^ 
= ^ ±W( ^ I +WEIWIE. (25) 

Due to the presence of the leak term on the left hand side of (24), the fixed point rate is stable only 
if the real parts of the eigenvalues Ai^2 are both smaller than 1. In the network with identical input 
statistics for all neurons the fiuctuating input is characterized by the same mean and variance (/i,CT^) 
for each neuron. For homogeneous neuron parameters the susceptibility Sa — S is hence the same for 
both populations a G {E,I}. If further the number of synaptic afferents is the same Kap — K for all 
populations, the eigenvalues can be expressed by those of the original connectivity matrix as 



Ai,2 ■hE±Iii±J(J_m_AiL\\j^^j^^ 



S{pL,a)K 



= ci ± y c| -I- JeiJie, 

where we defined the two parameters ci and C2 which control the location of the eigenvalues. In the 
left column of Figure 6 we keep Jei, Jie, and C2 constant and vary ci € [— C2, —y/c^ + JeiJie], where 
we choose the maximum value by the condition Ai < and the minimum value by the condition that 
Jee > and Jn < 0, leading to ci + C2 > and ci — C2 < 0, both fulfilled if — C2 < ci < C2. Varying C2 
in the right column of Figure 6, the bounds are given by the same condition that Jee > and Jn < 0, 
so C2 > 0, and the condition for the larger eigenvalue to stay below or equal 0, so C2 G [0, i/cf — JeiJie]- 
In order for the network to maintain similar mean activity, we choose the threshold of the neurons such 
that the cancellation condition = X^agI-B / x>(^"^)a^"^^ — is fulfilled for mp — 0.1. The resulting 
average activity is close to this desired value of 0.1 and agrees well to the analytical prediction (15), as 
shown in Figure 6A, B. 

The right-most point in both columns of Figure 6 where one eigenvalue vanishes Ai = 0, results in 
the same connectivity structure. This is the case for the connectivity with the symmetry Jee — Jie = J 
and Jn = Jei = —gJ (cf. Figure 4), because in this case the population averaged connectivity matrix 
has two linearly dependent rows, hence a vanishing determinant and thus an eigenvalue 0. As observed 
in Figure 6C,D at this point the absolute magnitude of correlations is largest. This is intuitively clear 
as the network has a degree of freedom in the direction of the eigenvector vi — {g, 1)'^ belonging to 
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the vanishing eigenvalue Ai = 0. In this direction the system effectively does not feel any negative 
feedback, so the evolution is as if the connectivity would be absent. Fluctuations in this direction hence 
become large and are only damped by the exponential relaxation of the neuronal dynamics, given by 
the left hand side of (24). The time constant of these fluctuations is then solely determined by the 
time constant of the single neurons, as seen in Figure 4B. From the coefRcients of the eigenvector we 
can further conclude that the fluctuations of the excitatory population are stronger by a factor g than 
those of the inhibitory population, explaining why cee > c//, and that both populations fluctuate in- 
phase, so cei > 0, (Figure 6C,D, right most point). Moving away from this point. Figure 6C,D both 
show that the magnitude of correlations decreases. Comparing the temporal structures of Figure 4B 
and Figure 5B shows that also the time scale of fluctuations increases. The two structural parameters 
ci and C2 affect the eigenvalues of the connectivity in a distinct manner. Changing ci merely shifts 
the real part of both eigenvalues, but leaves their relative distance constant, as seen in Figure 6E. For 
smaller values of ci the coupling among excitatory neurons becomes weaker, so their correlations are 
reduced. At the left most point in Figure 6C the coupling within the excitatory population vanishes, 
Jee = 0. Changing the parameter C2 has a qualitatively different effect on the eigenvalues, as seen 
in Figure 6F. At C2 = \/\JeiJie\, the two real eigenvalues merge and for smaller C2 they turn into a 
conjugate complex pair. At the left-most point Jee — Jii — 0, so both couplings within the populations 
vanish Jee = Jii = 0. The system then only has coupling from E to I and vice versa. The conjugate 
complex eigenvalues show that the population activity of the system has oscillatory solutions. This is also 
called the PING (pyramidal - inhibitory - gamma) mechanism of oscillations in the gamma-range [62] . 
Figure 6C,D show that for most connectivity structures the correlation structure is cei > cee > c//, 
in contrast to our previous finding [17], where we studied the symmetric case (the right-most point), 
at which the correlation structure is cee > cei > c//. The comparison of the direct simulation to the 
theoretical prediction (19) in Figure 6C,D shows that the theory yields an accurate prediction of the 
correlation structure for all connectivity structures considered here. 

4 Discussion 

The present work explains the observed pairwise correlations in a homogeneous random network of 
excitatory and inhibitory binary model neurons driven by an external population of finite size. On 
the methodological side the work required a minor extension of the seminal theory of correlations in 
stochastic binary networks by [33]. In particular, we improved the linearization procedure to account for 
the fluctuations due to the balanced synaptic noise [63]. This enables us to apply the existing theory 
[33] to neuron models with a hard threshold, without the need to assume the presence of additional noise 
local to each neuron. We further extend the framework by an iterative procedure taking into account 
the finitesize fluctuations in the mean-fleld solution to determine the working point (mean activity) of 
the network. We find that the iteration converges to predictions for the covariance with higher accuracy 
than the previous method. 

Equipped with these methods we investigate a network driven by correlated input due to shared 
afferents supplied by an external population. The analytical expressions for the covariances averaged 
over pairs of neurons show that correlations have two components that linearly superimpose, one caused 
by intrinsic fluctuations generated within the local network and one caused by fluctuations due to the 
external population. The size Nx of the external population controls the strength of the correlations 
in the external input. We find that this external input causes an offset of all pairwise correlations, 
which decreases with increasing external population size in proportion to the strength of the external 
correlations (ex 1/Nx)- The structure of correlations within the local network, i.e. the differences between 
correlations for pairs of neurons of different types, is mostly determined by the intrinsically generated 
fluctuations. These are proportional to the population-averaged variances qe and aj of the activity of 
the neurons in the local network. As a result, the structure of correlations is mostly independent of 
the external drive, even for the limiting case of an infinitely large external population Nx — >■ oo or if 
the external drive is replaced by a DC signal with the same mean. For the other extreme, when the 
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size of the external population equals the number of external afferents, Nx — K, all neurons receive an 
exactly identical external signal. We show that the mechanism of decorrelation [22, 17] still holds for 
these strongly correlated external signals. The resulting correlation within the network is much smaller 
than expected given the amount of common input. In contrast to an earlier explanation [22], which 
invokes the network's fast tracking of the external drive [50, 52] as the cause of small correlations, we 
here show that the cancellation of correlations between the inputs to pairs of neurons is equivalent to a 
suppression of fluctuations of the population-averaged input. This argument is in line with the earlier 
explanation that correlations are suppressed by negative feedback on the population level [17]. We further 
show that the cancellation of correlations does not uniquely determine the structure of correlations; there 
are different structures of correlations that lead to a cancellation of correlations between the summed 
inputs. The cancellation of input correlations therefore only constitutes a constraint for the pairwise 
correlations in the network. This constraint is trivially fulfilled under the assumption of perfect tracking, 
which assumes input fluctuations Sh to vanish completely. Therefore the attempt to obtain the structure 
of finite-sized correlations from fast tracking fails, although such a theory -by construction- reproduces 
the cancellation of input correlations [22]. Starting from the fast-tracking assumption, the resulting 
expressions for the correlations within the network [22, Supplementary, eqs. 38,39] are lacking the locally 
generated fluctuations as additional sources, shown here to be crucial for shaping the correlation structure. 
Note, however, that the intermediate result [22, Supplementary, eqs. 31,33] is still correct and identical 
to [33, eq. 6.8] and to (9). 

For a common but special choice of network connectivity where the synaptic weights depend only on 
the type of the source but not the target neuron, i.e. Jee = Jie and Jei = Jii [42], we show that the 
locally generated fluctuations and correlations are elevated and that the activity only loosely tracks the 
external input. The correlation structure exhibited is cee > cei > c//. To systematically investigate the 
dependence of the correlation structure on the network connectivity, it proves useful to parameterize the 
structure of the network by two measures differentially controlling the location of the eigenvalues of the 
connectivity matrix. We find that for a wide parameter regime the correlations change quantitatively, 
but the correlation structure cei > cee > c// remains invariant. The qualitative comparison with 
experimental observations of [49] hence only constrains the connectivity to be within the one or the other 
parameter regime. 

The presented theory is limited to sufficiently asynchronous and irregular network states. This lim- 
itation arises from the linearization procedure, which approximates the summed synaptic input by a 
Gaussian random variable. The deviations of the theory from direct simulations are stronger at lower 
mean activity, when the synaptic input fluctuates in the non-linear part of the effective transfer function. 
The best agreement of theory and simulation is hence obtained for a mean population activity close to 
i, where 1 means all neurons are active. For simplicity we consider in the theory networks where neu- 
rons have a fixed in-degree. In large homogeneous random networks this is often a good approximation. 



because the mean number of connections is pN ex N, and its standard deviation ^ Np{l — p) ex VJV 
declines relative to the mean. A comparison of this simplifying ansatz to a more realistic simulation with 
distributed in-degrees in Figure 5 shows good agreement; the results are not affected qualitatively. The 
dominant contribution to the observed deviations originates from the distribution of mean activities over 
neurons within each population due to differences in synaptic drive. This affects the population-averaged 
autocorrelations a^ no longer agreeing to TOq,(1 — TOq) in this case. An extension to distributed in-degrees 
can be done in the same manner as in the established mean-field theory [52]. 

One peculiarity arising in networks with a hard threshold is the independence of correlations of the 
synaptic coupling strength. This effect can be fully explained by the linearization procedure, because the 
susceptibility S* of a neuron decreases inversely proportional to the strength of the synaptic noise a. As 
the noise scales linearly in the synaptic amplitude, the effective linearized synaptic weight SJ is hence 
independent of J [63] . This also shows that a particular scaling of synaptic amplitudes with network size 
is not required to obtain small correlations in the balanced regime with irregular activity. 

The presented work is closely related to our previous work on the correlation structure in spiking 
neuronal networks [17] and indeed was triggered by the review process of the latter. In [17], we exclusively 
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studied the symmetric connectivity structure, where excitatory and inhibitory neurons receive the same 
input on average. The resuhs are quahtatively the same as those shown in Figure 4. A difference 
though is, that the external input in [17] is uncorrelated, whereas here it originates from a common finite 
population. The cancellation condition for input correlations, also observed in vivo [48], holds for spiking 
networks as well as for the binary networks studied here. The negative feedback suppressing fluctuations 
on the population level is the common underlying cause in both models. Future work needs to establish 
the formal relationship between the two network models. 

Our theory presents a step towards an understanding of how correlated neuronal activity in local 
cortical circuits is shaped by recurrence and inputs from other cortical and thalamic areas. The correlation 
between membrane potentials of pairs of neurons in somatosensory cortex of behaving mice is dominated 
by low-frequency oscillations during quiet wakefulness. If the animal starts whisking, these correlations 
significantly decrease, even if the sensory nerve fibers are cut, suggesting an internal change of brain state 
[5]. Our work suggests that such a dynamic reduction of correlation could come about by modulating 
the effective negative feedback in the network. A possible neural implementation is the increase of tonic 
drive to inhibitory interneurons. This hypothesis is in line with the observed faster fluctuations in the 
whisking state [5] . Further work is needed to verify if such a mechanism yields a quantitative explanation 
of the experimental observations. 

The network where the number of incoming external connections per neuron equals the size of the 
external population, cf. Figure 3 N,j: = K, can be regarded as a setting where all neurons receive an 
identical incoming stimulus. The correlations between this signal and the responses of neurons in the 
local network (Figure 3C) are smaller than in an unconnected population without local negative feedback. 
This can formally be seen from (24), because negative eigenvalues of the recurrent coupling dampen the 
population response of the system. This suppression of correlations between stimulus and local activity 
hence implies weaker responses of single neurons to the driving signal. Recent experiments have shown 
that only a sparse subset of around 10 percent of the neurons in SI of behaving mice responds to a sensory 
stimulus evoked by the active touch of a whisker with an object [4]. The subset of responding cells is 
determined by those neurons in which the cell specific combination of activated excitatory and inhibitory 
conductances drives the membrane potential above threshold. Our work suggests that negative feedback 
mediated among the layer 2/3 pyramidal cells, e.g. through local interneurons, should effectively reduce 
their correlated firing. In a biological network the negative feedback arrives with a synaptic delay and 
effectively reduces the low frequency content [17]. The response of the local activity is therefore expected 
to depend on the spectral properties of the stimulus. Intuitively one expects responses to better lock 
to the stimulus for fast and narrow transients with high frequency content. Further work is required to 
investigate this issue in more detail. 

A large number of previous studies on the dynamics of local cortical networks focuses on the effect 
of the local connectivity, but ignores the spatio-temporal structure of external inputs by assuming that 
neurons in the local network are independently driven by external (often Poissonian) sources. Our study 
shows that the input correlations of pairs of neurons in the local network are only weakly affected by 
additional correlations caused by shared external afferents: Even for the extreme case where all neurons 
in the network receive exactly identical external input (N^ ^ K), the input correlations are small and 
only slightly larger than those obtained for the case where neurons receive uncorrelated external input 
{Nx = 2iV; black curve in Figure 6C). One may therefore conclude that the approximation of uncorrelated 
external input is justified. In general, this may however be a hasty conclusion. Tiny changes in synaptic- 
input correlations have drastic effects, for example, on the power and reach of extracellular potentials 
[64]. For the modeling of extracellular potentials, knowledge of the spatio-temporal structure of inputs 
from remote areas is crucial. 

The theory of correlations in presence of externally impinging signals is a required building block to 
study correlation-sensitive synaptic plasticity [65] in recurrent networks. Understanding the emerging 
structure of correlations imposed by an external signal is the first step in predicting the connectivity 
patterns resulting from ongoing synaptic plasticity sensitive to those correlations. 
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